Field of the Invention
The invention relates to a method of registering first image data of a first stream of scans or images (e.g. of a first modality) with second image data of a second stream of scans or images (e.g. of a second modality). The first image data and the second image data comprise image information about a person or animal including image information about at least one organ of the person or animal. The shape of the organ is changing over time.
Description of Related Art
The human body (and also the body of animals) is in general not rigid and can move (i.e. change shape and/or position) during examinations and/or interventions. In addition, some organs move inside the body, even in cases when the body per-se as a whole is not moving. In case of specific organs, the organ movement relative to the body surface is cyclic and sometimes periodic or nearly periodic, for example organ movements caused by breathing, heartbeat and/or blood flow through vessels or organs.
In this description, organs performing repetitive, cyclic movements over time are called “moving organs”. Since organs neighboring an organ which undergoes a movement may also be passively moved, the term moving organs shall equally apply to the organ which causes the movement as well as to organs in the neighborhood which are passively moved by other organs.
In case of cyclically moving organs, the organ shape passes through a cycle of different shapes during the “movement cycle”: the organ shape and position changes starting from the initial location and shape (i.e. from the start of the cycle) and evolving continuously and steadily over a number of positions and shapes to the most deformed shape and/or most displaced position, where the movement process reverses in order to return back at the end of the cycle to the initial state. Then, a new cycle starts. The range or set of successive physiological conditions or stages (each one associated with a certain shape and position in space within the body), which a moving organ passes through during a movement cycle, is called “phase range” or “range of phase values”.
Breathing and heartbeat typically causes periodic or nearly periodic organ movements. By the general term “periodic” the two following different effects are addressed: (a) time-periodic repetition of the movement cycle within a certain, constant time-span which is called “time period” or simply “period”, i.e. the constant time duration between two successive movement repetitions; (b) shape-periodic repetition of a continuous process of shape changes within a movement cycle. In shape-periodic movements the duration of the movement cycle may or may not be constant, and in fact in anatomic processes it may vary significantly. However and independently of the time duration of a certain cycle, the shape changes of the organ during said cycle are repeated, i.e. the organ undergoes the same shapes during the movement cycle. The range of shapes which the organ successively passes through during such a movement cycle is called “shape pattern” or “shape range”. In contrast, in a perfect periodic movement such as a harmonic oscillation the time duration of all movement cycles is equally long and the shape pattern in all cycles is identical.
The typical cyclic organ movement has two well-defined extreme positions within the cycle: starting from an initial position and shape (which e.g. corresponds to complete exhalation or corresponds to the stage immediately after the heart end systole, etc.), the organ evolves over time to a point of maximum organ deformation and/or position change (which e.g. corresponds to complete inhalation or end of maximum of heart diastole, etc.). Then the organ movement direction is reversed and the organ shape and/or position returns back to its initial position and shape. At least in the case of heartbeat and breathing, such a cycle is then repeated again and again.
It is important to notice that under typical conditions (e.g. absence of severe pathology and/or external mechanical, electrical, pharmacological etc. influence), the time-period of organ movement is not strictly constant, but may vary significantly within physiological ranges (e.g. tachycardia vs. bradycardia, slow vs. fast breathing, exercising, stress, emotions, sleep . . . ). Thus, in terms of time, the organ movement is typically quasi-periodic, not precisely periodic. On the other hand, the shape range shows a much higher periodic stability: under typical conditions, organs pass through the same stages (the same shape and/or location) in each movement cycle. Therefore, organ movement can be seen as quasi-periodic in terms of time, but periodic in terms of shape and/or position.
In state-of-the-art imaging procedures a patient is examined with a plurality of medical imaging devices: CT (computed tomography), MRI (magnetic resonance imaging), PET (positron emission tomography), US (ultrasound) are just some common examples. Each such medical imaging device or instrument is designed for observing certain features of an organ. Such features can be inherently associated with the organ itself e.g. tissue density, water concentration, tissue reflectivity or permeability to sound waves etc., or can be indirect observations e.g. measurement of the concentration of a contrast agent, radio-pharmacon etc. Frequently a plurality of different imaging protocols can be executed with one imaging device, e.g. native or contrast agent CT, single or multi-parameter MRI etc. In other words, not only can an organ be examined by different devices, but also several different types of scans can be performed with the same device, e.g. an MRI can be used to measure water density in T1 or T2 scans, but also diffusion or perfusion coefficient, concentration of contrast agent, etc. In addition, sometimes the same instrument can be used in different space-sampling modes, e.g., one-dimensional (1D), two-dimensional (2D) and three-dimensional (3D) ultrasound. Furthermore, acquisitions of image data can be performed at a specific point in time (e.g. photographs, screenshots, single images, single tomographies etc.) or over a time interval (films, cinematographic loops, 4D CT, 4D MRI etc.).
To avoid confusion we call “modality” the feature of an organ observed with a certain device. Obviously several features can be observed with the same or different instruments or devices. Thus the term modality is associated in our case with a feature rather than with a specific instrument type.
Generally speaking, an acquisition of images samples a body region, an organ or a part of an organ instantaneously (single shot at a certain point of time) or repeatedly (repeated shots at several sequent time points such a film or loops). We call a “scan” a set of values acquired by an image acquisition device and related to a feature of an organ at a certain single point of time in 2D or 3D space. The space thereby is typically discrete (digital images consisting of pixels), but could be also continuous as e.g. in the case of an X-ray film. A more general expression for “scan” is “image”, since it also covers the case that the image is generated otherwise than by an image acquisition device (such as an US or MR device), in particular by interpolating scans and/or images or by transforming a scan or image.
In the case that a single value is associated to every space position, the scan or image is called “monomodal scan”. In contrast, if multiple values are associated to each space position, the scan or image is called “multimodal scan”. Thus, the terms scan and image address both 2D images such as X-rays or ultrasounds, and 3D volumes e.g. tomographies. When mentioning just scan or image, one addresses both, monomodal as well as multimodal scans or images. To be precise, the term “certain single point of time” also includes small time intervals, since the picture elements (pixels) of a scan can be acquired e.g. one after the other or line after line.
A series of scans acquired over consecutive, discrete time-steps is called “stream”. A stream can also include other images, in particular images generated by interpolation or transformation of other images. As such, streams have one set of variables describing the position(s) in space and an additional variable describing the point in time. Streams include cine-loops, 4D-CT, 4D-MRI etc. The term “consecutive” includes the case that there is at least one time interval in between the discrete time-steps and that no image data of this particular stream are acquired during the time interval. The image acquisition device may perform other tasks during the time interval, such as acquiring image data of another stream, processing image data and/or transmitting image data. However, it is also possible that the discrete time-steps of acquiring in each case a single scan or image of the stream form a consecutive series of time-steps with no time intervals in between.
Summarizing, but only as an illustrating example, a patient can be scanned by a plurality of devices generating a plurality of modalities at a plurality of occasions at different points of time, e.g. over a time-span of seconds, days, months or years. Frequently, medical procedures for diagnosis or therapy require an association of such datasets acquired with the same device or with different devices and optionally at different times. In particular, the result of a first examination of a patient is to be associated with the corresponding result of a second examination of the patient in a way that corresponding anatomic locations are associated, or overleaped, within the different scans or images. Without losing generality, we refer to “registration” as the creation of location-to-location associations between all anatomically corresponding locations of two individual images. In case of more than two images, the process is pairwise repeated and partial results are concatenated. Today, a plurality of methods is known to enable rigid and non-rigid registration between corresponding images. Obviously registrations can be performed between images of the same imaging modality (e.g. overlap of older and actual CTs), or between different imaging modalities (e.g. overlap MRI and CT). Such multi-modal registrations are also called “fusion imaging”. For example, CT scans and ultrasound scans can be registered to form fusion imaging scans of a combined modality. In our case we regard mono-modal registrations to be a special case of the more general multi-modal registrations and therefore we refer as fusion imaging to monomodal and multimodal registrations equally.